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Abstract 

We measure trends in the diffusion of misinformation on Facebook and Twitter between Jan¬ 
uary 2015 and July 2018. We focus on stories from 570 sites that have been identified as 
producers of false stories. Interactions with these sites on both Facebook and Twitter rose 
steadily through the end of 2016. Interactions then fell sharply on Facebook while they con¬ 
tinued to rise on Twitter, with the ratio of Facebook engagements to Twitter shares falling by 
approximately 60 percent. We see no similar pattern for other news, business, or culture sites, 
where interactions have been relatively stable over time and have followed similar trends on 
the two platforms both before and after the election. 
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1 Introduction 


Misinformation on social media has caused widespread alarm in recent years. A substantial number 
of U.S. adults were exposed to false news stories prior to the 2016 election, and post-election 
surveys suggest that many people who read such stories believed them to be true (Silverman and 
Singer-Vine 2016; Allcott and Gentzkow 2017; Guess et al. 2018). Many argue that false news 
stories played a major role in the 2016 election (for example, Olson 2016; Parkinson 2016; Read 
2016; Gunther et al. 2018), and in the ongoing political divisions and crises that have followed it 
(for example, Spohr 2017; Azzimonti and Fernandes 2018; Tharoor 2018). Numerous efforts have 
been made to respond to the threat of false news stories, including educational and other initiatives 
by civil society organizations, hearings and legal action by regulators, and a range of algorithmic, 
design, and policy changes made by Facebook and other social media companies. 

Evidence on whether these efforts have been effective—or how the scale of the misinformation 
problem is evolving more broadly—remains limited. A recent study argues that false stories remain 
a problem on Facebook even after changes to its news feed algorithm in early 2018 (Newswhip 
2018). The study reports that the 26th and 38th most engaging stories on Facebook in the two 
months after the changes were from fake news websites. Many articles that have been rated as 
false by major fact-checking organizations have not been flagged in Facebook’s system, and two 
major fake news sites have seen little or no decline in Facebook engagements since early 2016 
(Funke 2018). Facebook’s now-discontinued strategy of flagging inaccurate stories as “Disputed” 
can modestly lower the perceived accuracy of flagged headlines (Blair et al. 2017), though some 
research suggests that the presence of warnings can cause untagged false stories to be seen as 
more accurate (Pennycook and Rand 2017). Media commentators have argued that efforts to fight 
misinformation through fact-checking are “not working” (Levin 2017) and that misinformation 
overall is “becoming unstoppable” (Ghosh and Scott 2018). 

In this paper, we present new evidence on the volume of misinformation circulated on social 
media from January 2015 to July 2018. We assemble a list of 570 sites identified as sources of false 
stories in a set of five previous studies and online lists. We refer to these collectively as fake news 
sites. We measure the volume of Facebook engagements and Twitter shares for all stories on these 
sites by month. As points of comparison, we also measure the same outcomes for stories on (i) a 
set of major news sites; (ii) a set of small news sites not identified as producing misinformation; 
and (iii) a set of sites covering business and culture topics. 

The results show that interactions with the fake news sites in our database rose steadily on both 
Facebook and Twitter from early 2015 to the months just after the 2016 election. Interactions then 
declined by more than half on Facebook, while they continued to rise on Twitter. The ratio of 
Facebook engagements to Twitter shares was roughly steady at around 40:1 from the beginning 
of our period to late 2016, then fell to roughly 15:1 by the end of our sample period. In contrast, 
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interactions with major news sites, small news sites, and business and culture sites have all re¬ 
mained relatively stable over time, and have followed similar trends on Facebook and Twitter both 
before and after the 2016 election. While this evidence is far from definitive, we see it as consis¬ 
tent with the view that the overall magnitude of the misinformation problem may have declined, at 
least temporarily, and that efforts by Facebook following the 2016 election to limit the diffusion of 
misinformation may have had a meaningful impact 

The results also show that the absolute level of interaction with misinformation remains high, 
and that Facebook continues to play a particularly important role in its diffusion. In the period 
around the election, fake news sites received almost as many Facebook engagements as the 38 
major news sites in our sample. Even after the sharp drop following the election, Facebook en¬ 
gagements of fake news sites still average roughly 70 million per month. 

Our evidence is subject to many important caveats and must be interpreted with caution. This is 
particularly true for the raw trends in interactions. While we have attempted to make our database 
of false stories as comprehensive as possible, it is likely far from complete, and many factors could 
generate selection biases that vary over time. The raw decline in Facebook engagements may 
partly reflect the under-sampling of sites that could have entered or gained popularity later in our 
sample period, as well as efforts by producers of misinformation to evade detection on Facebook by 
changing their domain names. It may also reflect changes over time in demand for highly partisan 
political content that would have existed absent efforts to fight misinformation, and could reverse 
in the future, for example in the run-up to future elections. 

We see the comparison of Facebook engagements to Twitter shares as potentially more infor¬ 
mative. If the design of these platforms and the behavior of their users were stable over time, we 
might expect sample selection biases or demand changes to have similar proportional effects, and 
thus leave the ratio of Facebook engagements to Twitter shares roughly unchanged. For example, 
we might expect producers changing domain names to evade detection to produce similar declines 
in our measured interactions on both platforms. The fact that Facebook engagements and Twit¬ 
ter shares follow similar trends prior to late 2016 and for the non-fake-news sites in our data, but 
diverge sharply for fake news sites following the election, suggests that some factor has slowed 
the relative diffusion of misinformation on Facebook. The suite of policy and algorithmic changes 
made by Facebook following the election seems like a plausible candidate. 

However, even the relative comparison of the platforms is only suggestive. Both Facebook 
and Twitter have made changes to their platforms, and so at best this measure captures the relative 
effect of the former compared to the latter. Engagements on Facebook affect sharing on Twitter 
and vice versa. The selection of stories into our database could for various reasons differentially 
favor the kinds of stories likely to be shared on one platform or the other, and this selection could 
vary over time. Demand changes need not have the same proportional effect on the two platforms. 
Some of these factors would tend to attenuate changes in the Facebook-Twitter ratio, leading our 
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results to be conservative, but others could produce a spurious decrease over time. 

In the appendix, we show that our qualitative results survive a set of robustness checks intended 
to partially address potential sample selection biases. These checks include: (i) focusing on sites 
identified as fake in multiple lists; (ii) excluding sites from each of our five lists in turn, (iii) looking 
at sites that were active in different periods; (iv) excluding potential outliers and looking at sites of 
different sizes; and (v) looking at sites with different likelihoods to publish misinformation. 

2 Background 

Both Facebook and Twitter have taken steps to reduce the circulation of misinformation on their 
platforms. In the appendix, we list twelve such announcements by Facebook and five by Twitter 
since the 2016 election. Broadly, the platforms have taken three types of actions to limit misinfor¬ 
mation. First, they have limited its supply, by blocking ads from pages that repeatedly share false 
stories and removing accounts that violate community standards. Second, they have introduced 
features such as “disputed” flags or “related articles” that provide corrective information related 
to a false story. Third, they have changed their algorithms to de-prioritize false stories in favor of 
news from trustworthy publications and posts from friends and family. 

Legislators are also taking action. For example, Connecticut, New Mexico, Rhode Island, and 
Washington passed laws in 2017 encouraging media literacy and digital citizenship (Zubrzycki 
2017). Executives from Facebook, Google, and Twitter have been asked to testify before various 
congressional committees about their efforts to combat misinformation (Shaban et al. 2017; Pop- 
ken 2018). Although there has been no major national legislation, this testimony may have raised 
public awareness. 

Finally, civil society organizations also play an important role. For example, the News Literacy 
Project provides non-partisan educational materials to help teachers educate students to evaluate 
the credibility of information; demand for its materials has grown substantially in the past few 
years (Strauss 2018). In 2017, the newly established News Integrity Initiative (Nil) made ten grants 
totaling $1.8 million to help build trust between newsrooms and the public, make newsrooms more 
diverse and inclusive, and make public conversations less polarized (Owen 2017). 

3 Data 

We compile a list of sites producing false news stories by combining five previous lists: (i) a re¬ 
search project by Grinberg et al. (2018, 490 sites); (ii) PolitiFact’s article titled “PolitiFact’s guide 
to fake news websites and what they peddle” (Gillin 2017, 325 sites); (iii) three articles by Buz- 
zFeed on fake news (Silverman 2016; Silverman et al. 2017a; Silverman et al. 2017b; 223 sites); 
(iv) a research project by Guess et al. (2018, 92 sites); and (v) FactCheck’s article titled “Websites 
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that post fake and satirical stories” (Schaedel 2017, 61 sites). Politifact and FactCheck are indepen¬ 
dent journalistic fact-checking websites, while BuzzFeed similarly applies journalistic standards to 
evaluating whether articles are true or false. The two lists from research projects originally derive 
from subsets of the other three, plus Snopes.com, another independent fact-checking site, and lists 
assembled by blogger Brayton (2016) and media studies scholar Zimdars (2016). The union of 
these five lists is our set of fake news sites. 

PolitiFact and FactCheck work directly with Facebook to evaluate the veracity of stories flagged 
by Facebook users as potentially false. Thus, these lists comprise fake news sites that Facebook 
is likely to be aware are fake. As a result, our results may be weighted toward diffusion of mis¬ 
information that Facebook is aware of, and may not fully capture trends in misinformation that 
Facebook is not aware of It is difficult to assess how large this latter group might be. Our list 
almost certainly includes the most important providers of false stories, as Facebook users can flag 
any and all questionable articles for review. On the other hand, the list might exclude a large tail 
of small sites producing false stories. 

Combining these five lists yields a total of 673 unique sites. We have data for 570 of them. We 
report in the appendix the names and original lists of 50 largest sites in terms of total Facebook 
engagements plus Twitter shares during the sample period. In our robustness checks, we consider 
alternative rules for selecting the set of sites. 

Our sets of comparison sites are defined based on category-level web traffic rankings from 
Alexa. 1 Alexa measures web traffic using its global traffic panel, a sample of millions of Internet 
users who have installed browser extensions allowing their browsing data to be recorded, plus data 
from websites that use Alexa to measure their traffic. It then ranks sites based on a combined mea¬ 
sure of unique visitors and pageviews. We define major news sites to be the top 100 sites in Alexa’s 
News category. We define small news sites to be the sites ranked 401-500 in the News category. 
We define business and culture sites to be the top 50 sites in each of the Arts, Business, Health, 
Recreation, and Sports categories. For each of these groups, we omit from our sample government 
websites, databases, sites that do not mainly produce news or similar content, international sites 
whose audiences are primarily outside the U.S., and sites that are included in our list of fake news 
sites. Our final sample includes 38 major news sites, 78 small news sites, and 54 business and 
culture sites. 

We gather monthly Facebook engagements and Twitter shares of all articles published on these 
sites from January 2015 to July 2018 from BuzzSumo (www.buzzsumo.com). BuzzSumo is a 
commercial content database that tracks the volume of user interactions with internet content on 
Facebook, Twitter, and other social media platforms, using data available from the platforms’ appli¬ 
cation programming interfaces (APIs). We use BuzzSumo’s data on total Facebook engagements 
and total Twitter shares by originating website and month. Facebook engagements are defined 
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as the sum of shares, comments, and reactions such as "likes." We have data for 570 out of 673 
fake news sites in our list and all sites in the comparison groups. We sum the monthly Facebook 
engagements and Twitter shares of articles from all sites in each category and then average by 
quarter. 

4 Results 

Figure [T] shows trends in the number of Facebook engagements and Twitter shares of stories from 
each category of site. Interactions for major news sites, small news sites, and business and culture 
sites have remained relatively stable during the past two years, and follow similar trends on Face- 
book and Twitter. Both platforms show a modest upward trend for major news and small news 
sites, and a modest downward trend for business and culture sites. In contrast, interactions with 
fake news have changed more dramatically over time, and these changes are very different on the 
two platforms. Fake news interactions increased steadily on both platforms from the beginning of 
2015 up to the 2016 election. Following the election, however, Facebook engagements fell sharply 
(declining by more than 50 percent), while shares on Twitter continued to increase. 

Figure [2] shows our main result: trends in the ratio of Facebook engagements to Twitter shares. 
The ratios have been relatively stable for major news, small news, and business and culture sites. 
For fake news sites, however, the ratio has declined sharply, from around 45:1 during the election 
to around 15:1 two years later. 

While these results suggest that the circulation of fake news on Facebook has declined in both 
absolute and relative terms, it is important to emphasize that the absolute quantity of fake news 
interactions on both platforms remains large, and that Facebook in particular has played an outsized 
role in its diffusion. Figure [T] shows that Facebook engagements fell from a peak of roughly 200 
million per month at the end of 2016 to roughly 70 million per month at the end of our sample 
period. As a point of comparison, the 38 major news sites in the top left panel—including the New 
York Times, Wall Street Journal, CNN, Fox News, etc.—typically garner about 200-250 million 
Facebook engagements per month. On Twitter, fake news shares have been in the 4-6 million per 
month range since the end of 2016, compared to roughly 20 million per month for the major news 
sites. 

We report a number of robustness checks in the appendix designed to address concerns about 
selection into our sample of sites. First, we restrict to sites that are identified as fake news sites by 
at least two or three of our original five lists, which leaves 116 and 19 sites, respectively. Second, 
given that people might disagree with any one particular study’s list of fake news sites, we run 
five additional analyses, each excluding fake news sites identified exclusively by one of our five 
lists. Third, we focus on sites that started active operations after November 2016, sites that were 
still in active operation as of July 2018, and sites that were in active operation from August 2015 
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to July 2018, which leaves 226, 215, and 82 sites respectively. (Active operation is defined to 
be a global traffic rank reported by Alexa of at least one million.) Fourth, we exclude the five 
largest sites in terms of total interactions to ensure the trend is not driven solely by outliers. We 
also look at sites in the first decile and sites in the bottom nine deciles separately to see if the 
trend holds for both large sites and small sites. Fifth, Grinberg et al. (2018) provide three lists of 
sites classified by different likelihoods to publish misinformation. We look at each of these lists 
separately. Our main qualitative conclusions remain consistent across these checks, though the 
exact size and shape of the trends vary. Finally, we present an alternative comparison group: a 
small set of politically focused sites such as Politico and The Hill. These sites do see a decline in 
engagements on Facebook relative to Twitter, but it mainly occurred in late-2015. 
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Figure 1: Engagement on Facebook and Twitter 
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Notes: This figure shows monthly Facebook engagements and Twitter shares of all articles published on sites in 
different categories averaged by quarter. Data comes from BuzzSumo. Major News Sites include 38 sites selected 
from the top 100 sites in Alexa’s News category. Small News Sites include 78 sites selected from the sites ranking 
401-500 in the News category. Business and Culture Sites include 54 sites selected from the top 50 sites in each of 
the Arts, Business, Health, Recreation, and Sports categories. Fake News Sites include 570 sites assembled from five 
lists. The complete lists can be found in the appendix. 
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Figure 2: Relative Engagement on Facebook 
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Notes: This figure shows the ratio of monthly Facebook engagements over Twitter shares of all articles published on 
sites in different categories averaged by quarter. Data comes from BuzzSumo. Major News Sites include 38 sites 
selected from the top 100 sites in Alexa’s News category. Small News Sites include 78 sites selected from the sites 
ranking 401-500 in the News category. Business and Culture Sites include 54 sites selected from the top 50 sites in 
each of the Arts, Business, Health, Recreation, and Sports categories. Fake News Sites include 570 sites assembled 
from five lists. The complete lists can be found in the appendix. 
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